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Input devices 
A 



124 



121 



202 



204 



206 



208 



210 



212 



126 



128 



122 



1 



Microphone 



Video camera 



Body temp, 
sensor 



EKG/EEG 
sensor 



Heart rate 
monitor 



Blood pressure 
monitor 



Blood O2 sensor 



Ambient temp, 
sensor 



Various user 
sensor devices 



Various 
environment 
sensor devices 



Various user 
input devices 



CPU 



'234 



I 1 

it 



232 




120 Output devices 



248 
230 



214 



Audio input 
CS 



216 



Video input CS 



218 



Body temp. CS 



220 



Heart rate CS 



222 



Blood pressure 
CS 



224 



Blood O2 CS 



226 



Other CS's 



CM 



240 

"1 



•H '-B 

I 



Formatter 
and visual 
display 
selector 



242 



O 

o 



244 



Other CS's 
^ 



Application program 



236 



Browser program 



238 



Fig. 2 



Head-mounted 
display 



130 



Flat panel 
display 



132 



Earpiece 
speaker 



136 



Tactile output 
device 



252 



Telecom, 
transceiver 



254 



Speaker 



250 



Defibrillator 



138 



Various user 
output devices 



Code 
module A 
(CS) 



Code 
module B 
(CC) 



Code 
module C 
(CS/CC) 



Code 
module D 
(CS/CC) 



CM 



Fig. 3 




300 



Initialize CS and/or CC 
for theme set for user 



402 



At CS, obtain data signals from input devices 



At CS, process data signals to produce 
attributes representing user's context, 
including adding required data fields, such as 
attribute name, timestamp, units, etc. 



At CS, provide attribute to CM for storage 



At CC, request from CM and receive attributes 



404 



406 



408 



410 



At CC, process attributes to determine appropriate output 



Provide output signal to user, other entity or other process 



412 



414 



Fig. 4 



Launch at start-up 



502 



Periodically receive GPS data stream 



Parse and store GPS data 



Receive 
^request for latitude,^^N 
4ongitude or altitude, 
^attribute?. 



Provide requested attribute value(s), 
with Timestamp, uncertainty, units, etc. 



504 



506 



510 



Fig. 5 A 



Name 



Latitude (value) 



Uncertainty 



Timestamp 



Units 



Format version/flags 



521 
522 
524 
526 
528 
530 



Fig. 5B 



Register for latitude, longitude and altitude attributes 



602 



604 




Normal operation? 



606 



Use default attribute mediator 



Reevaluate all latitude attributes 



'608 



Select lowest uncertai 


r 

nty attribute mediator 




r 


Determine CS providing resulting attributes 


^ 


< 



•610 



612 



Request latitude, longitude and altitude attributes from CS/CM 



616 




^otify feature active^. 



Receive input derfining boundary 



•618 



Provide event parameters to CM regarding boundary transversal 



622 

Receive 
response from CM to^^;>]!i 
established event?. 



Notify user that boundary has been crossed 



624 



614 



620 




600 



Fig. 6 



User. 

Desired_j)rivacy_level 

Interruptibiiity 

Speed 

Direction 

Acceleration 

Availability. 

Cognitive_availability 

Tactile_availability 

Manual_availability 

Visual_availability 

Oral^availability 

Auditory__availability 
Proximity <Item or place name> 
Mood. 

Happiness 

Sadness 

Anger 

Frustration 

Confusion 
Activity. 

Driving 

Eating 

Running 

Sleeping 

Talking 

Typing 

Walking 
Location. 

Place_name 

Latitude 

Longitude 

Altitude 

Room 

Floor 

Building 

Address 

Street 

City 

County 

State 

Country 

Postal_Code 
Destination, (same as User.Location.) 
Physiology- 
Pulse 

Body_temperature 

Blood_j)ressure 

Respiration 
Person. <name or ID>. (same as User.) 
Platform. 
UI. 

Qral_input_device_availability 

Manual_input_device_availability 

Tactile_output__device_availability 

Visual_output_device_availability 

Auditory_output_device_availability 



Fig. 8 



Platform, (continued) 
CPU. 

Load 

Speed 
Memory. 

Total_capacity 

Used 
Storage. 

Total__capacity 

Used 
Cormectivity. 

Connection_status 

Cormection_speed 

Connection_type/device 

Connection_activity 
Power. 

Power_source 

Power_level 
Environment. 
People. 

Nearest 

Number_present 
Number_close 
Local. 

Time 
Date 

Temperature 

Pressure 

Wind_speed 

Wind_direction 

Absolute_humidity 

High_forecast_temperature 

Low_forecast_temperature 

People_present 

Ambient_noise__level 

Ambientjight_level 

Days.<previous or future>. 

High_temperature 

Low_temperature 

Precipitation_type 

Precipitation_amount 
Place.<place name>. (same as Environment. Local) 
Application. 
Mail. 

Available 

New_mes sages_waiting 
Messages_waiting_to_be_sent 
Phone. 

Available 

In_use 

On/off 

Notification_mechanism 
Calljncoming 
Caller_ID 
Soundjrecorder. 
Available 
Recording 



User Setting Computer Fig. 9 

Mental Context Power 

Meaning Configuration 

Cognition User Input Systems 

Divided User Attention Hand/Haptic 

Task Switching Keyboard/Keystrokes 

Background Awareness Voice/Audio 

Solitude Eye Tracking 

Privacy Cursors 

Desired Privacy Axis 

Perceived Privacy Resolution 

Social Context Selection 

Affect Invocation 

Physical Situation Accelerators 

Body Output Systems 

Biometrics Visual 

Posture Resolution 

Motion Audio 

Physical Encumberment Public/Private 

Senses Haptic 

Eyes External Resources 

Ears I/O devices 

Tactile Connectivity 
Hands Data 

Nose Quantity/State 

Tongue Urgency/Importance 

Workload demands/effects Use of Prominence 

Interaction with computer devices Modality 

Interaction with people Sensitivity/Purpose 

Physical Health Privacy Issues 

Environment Use of Association 

Time/Space Use of Safety 

Objects Source/Ownership 

Persons Types 

Audience/Privacy Availability User generated 

Scope of Disclosure Other computers or people 

Hardware affinity for privacy Sensor 

Privacy Indicator for User PC State 

Privacy Indicator for Public Use of Association 
Watching Indicator 
Being Observed Indicator 
Ambient Interference 
Visual 
Audio 
Tactile 
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Example Current Theme Selection 



Select A New Current Theme 



1150- 



^Starting The Day 

-Driving Home 
-At Home 

Work-Related Group 
-Driving To Work 
-Arriving At Work > 1 1 55 
-At Work 

: J 

Entertainment-Related Group 




OK 



1157 



Cancel 

J — 



Figure IIM 



Starting The Day 

1150^ At Home 

.Eating Breakfast 



Example Current Theme Set Modification 

Current Theme Set 



Add Theme 



1157 



A. 



Remove Theme 



Figure UN 



Example Current Theme Layout Selection 

Select A New Current Theme Layout 

Starting The Day theme 
r Starting The Day - default layout 

1 1 60 <| Starting The Day - vacation 

L Starting The Day - responsible for dropping off kids 



OK 



Cancel 



1157 



Figure llO 





1302 

s 


1304 

\ Example Theme Data Structure 






DrivitiP To W^orlc 


1310-^ 




Intent Is To Organize Roth Dnvins Info & Info About Work Arrival 


1315"^ 


L>CCm ILy 




1320'-N^ 




Arrp'5<!' Sielf + Familv + Friends + Source' VFodification" Self' Conv* None* 

X^V^bOO. O^XX ' X CUilXXy ' X XiWXXVXO ■ UVJUXW^^ XVXVyVXXXXV/CVU.VXX. UV^XXj \^\J^ J ^ X^V^XlVifj . . . 


1325'-V> 


JT 1 1 V ci\jy 


Private 


1330'-N^ 


Pri nritv 




1335''X^ 


Group-ID 


23 




Time-Active 


Mon-Fri, 06:00-11:00 


1345^X> 


Source 


BCD Motor Company 


1350^V^ 


Theme-Content 


"BCD Motor logo.gif 


1355''X^ 


New-Attributes 


DTW-BCD.driving-direction: (source = "BCD-DTW CS"); . . . 


1360^X> 


Theme-Matching 


user. activity. driving = true (required); environment. local. time = 08:00-09:00 (optional); 
DTW-BCD.driving-direction = NOT "South" (required), . . . 




Theme-Logic 


IF <user. activity. driving> AND user. location. latitude > recent.user. location. latitude 

THEN DTW-BCD.driving-direction = '^orth"; 
IF DTW-BCD.driving-direction = 'East" THEN Alert("You're Lost Again! !"); 
SET user, location [Access] = Family & Co-workers; . . . 


1370'-\^ 


Attribute-Set 


<link^>; • • • <link>> 




: \ \ 



r Attribute 1 



Attribute N 



Name 


user, activity, driving 


Value 


True 


Uncertainty 


High 


Timestamp 


07:32 03/24/XX 


Units 


boolean 


Access 


Family 





Name 


DTW-BCD.driving-direction 


Value 


West 


Uncertainty 


None 


Timestamp 


08:13 03/24/XX 


Units 


North, North by Northeast, . . . 


Access 


Self 





Fig. 13 
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Fig. 15 



1500 

Theme usage 
routine 



1 


f 1505 


Receive indications of currently available themes 
and their associated theme information 




r 1510 


Retrieve current context information 


► 


r 1515 




Determine themes that match current context 


1530 1 


r 1520 



Present matching or all 


^ Yes 


themes to user 






r 1535 




Receive indication of 




selected theme 





User selection? 



No 



1525 



Select highest priority 
matching theme 



1540 



Generate appropriate 
response to selected theme 



1545 



Detect changes to context or user indication 



1555 



V 1550 




User indication? 



1565 





Present current context 


^ Yes 




information to user 








r 1570 






Receive indication of 




< — 


context changes 







Yes 




f 1560 



Change context 
information? 



.No 




1595 



END 



Yes 1575 



Exit? 



1580 




No 


Provide indicated 


< 1 




functionality to user 







1540 

Response generator- 
subroutine 



Fig. 16 



1605 



Receive indication of selected theme 
and of associated theme information 



1615 

Select default theme layout 
if multiple are associated 



1620 



Retrieve content specified 
by theme layout 



1625 



Perform data logic specified 
by theme layout 



Yes. 



i 1610 

Associated theme 
layout(s)? 



No 



1630 



Retrieve user preference 
information 



± 1635 

Present content in manner specified by presentation logic 
of theme layout and in a maimer consistent with 
preference information 



1640 



Other response 
specified by theme 
layout? 



^r_^l645 



Provide other response as 
appropriate 



yr 1695 



Return 



1700 

Theme creator/ 
modifier routine 











r 1705 




Receive indication of theme 
to be created or modified 


1715 




r 1710 



Fig. 17 



1720 




Display theme to user 



1730 



Receive indication firom user to modify 
specified properties of theme or 
information associated with theme 



1740 



Present theme layout to user 
in visual editor 



1735 



LYeSy 



Modify theme 
layout? 





No 




r 1750 



1755 



Modify property? 



.Yes 



1745 



Modify theme layout as 
indicated by user 



Modify theme property as 
indicated by user 



No 



1760 



Modify theme attribute, 
theme CS, or theme CC as 
indicated by user 



1765 



Yes. 



More changes to 
theme? 



No 



1770 



Store theme and 
theme-related information 



Yes. 



V 1775 



More themes? 



1795 



.No 



END 



1800 

Theme distributor 
routine 



Fig. 18 



1805 



Receive indications of 
available themes 



1810 



Retrieve indications of 
appropriate users for the 
available themes 



1815 



Receive request for theme(s) 
or indication of accessible 
user 



1835 



Retrieve information about 
user from database or user's 
computing device 



1820 



1825 



No. 



Request? 



.Yes 



Determine if user has 
provided appropriate access 
information 



1830 



1840 



Determine themes that are 
appropriate for user 



Yes, 



Appropriate 
access? 



1845 




Determine if any needed 
payment information has 
been provided 



1850 



Payment 
provided? 



.No 



Yes 



1855 



Gather theme-related information 
associated with requested or determined 
themes, including the themes 



1860 



Send gathered information 
to requesting or indicated 
user 



1865 



Yes. 



Continue? 





No 




r 1895 



END 



1900 

Theme receiver 
routine J 



1905 



Receive user request for 
theme or indication of sent 
theme 



No. 





r 1940 


Receive indication of sent 
theme 




r 1945 



No. 



Payment 
information 
needed? 



> 


Yes 

r 1950 


Send payment information 


1 


r 1955 



Payment accepted? 



.No 



Yes 



1960 



Receive theme and any 
associated theme-related 
information 



1965 



Store received information, 
and load for use if 
immediately useful 



Yes. 



Fig. 19 



V 1910 



1915 



Request? 



.Yes 



V 1922 



Indicate failure 
to user 



\f 1970 



Continue? 



Attempt to determine theme 
server that can provide 
requested theme 



1920 



No, 



Theme server 
found? 



Yes 



1925 



Receive appropriate access 
information for theme and 
theme server, either from 
user or from storage 



1930 



Receive indication of 
payment mechanism if 
needed, either from user or 
from storage 



1935 



Send request for theme to 
theme server, including any 
access information and 
payment information 





No 


> 


f 1995 



END 



2000 



Automated theme 
customizer routine 



No 



2005 



Receive indications of available themes 




r 2010 



Repeatedly monitor user actions, 
including interactions with themes, 
modifications to themes, and explicit 
changes to the current theme or the 
current context 





r 2015 


Analyze monitored 
patterns ( 


actions to determine 
)f actions 



2020 



Any patterns above relevance 
threshold? 



Do automated modification? 



Fig. 20 





Yes 

f 2025 


Retrieve user preference information 




r 2030 


Determine modifications to some or all 
themes that are consistent with patterns 
above the relevance threshold and with 
user preferences 




f 2035 



2050 



1 


Yes 

r 2040 


Modify themes as determined 




r 2045 


Notify user of modifications 


1 


r 2065 



Yes 



Contmue? 





No 




f 2095 



No ^ 


Present suggested modifications to the 




user 






r 2055 




Receive indications of accepted 
modifications 






r 2060 




Modify themes as indicated 


>< 



END 



